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ABSTRACT 



This study investigated Hong Kong elementary school 
teachers' and students' attitudes toward cooperative learning. Teachers from 
one school completed a two-part questionnaire. Part 1 asked for personal 
information. Part 2 had eight sections that discussed: organizational 
strategies used in the classroom; frequency of using classroom organizational 
strategies in four core subjects; the basis of grouping for small group work- 
group size; how group tasks were organized; social skills of working in 
groups; factors that might influence respondents' decisions to use various 
classroom organizational strategies; and examples of successful and 
unsuccessful use of group work in teaching. Four respondents also completed 
interviews after completing the questionnaire. Students from the same school 
responded in writing to a pair of cartoon pictures, discussing what they 
thought groups of children in pictures with and without teachers were saying. 
Results indicated that teachers' attitudes toward group work were generally 
positive. Relatively more teachers spent half of their teaching time in group 
work. Teachers had reservations about using group work because there were so 
many school restraints (e.g., class size and limited time) . Not all teachers 
used heterogeneous grouping or taught group social skills. The childrens' 
drawings suggested that teachers being present did not affect the kinds of 
conversations children had. Children appeared to prefer group work. (Contains 
58 references.) (SM) 
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Cooperative learning in Hong Kong schools: attitudes of teachers and 
pupils towards cooperative group work 

Kam Wing Chan 
Maurice Galton 

Background 

Group work as a teaching strategy is popular in western countries. In Hong Kong 
group work was seldom employed until 1975 when Activity Approach was 
introduced into the primary curriculum. Traditional teacher-centred approach was the 
mainstream before then. Activity Approach is pupil-centred and is characterized by 
the use of group work in the learning process. Despite the promotion of Activity 
Approach by the Hong Kong Education Department, it was not well received due to 
various reasons. It was also difficult to convince the parents that Activity Approach 
was equally effective as the traditional approach, if not better, as the majority of 
Chinese parents believed that children would not learn well by participating in group 
work. The parents’ socio-cultural background that encouraged passive and reserved 
behaviours made them think that this approach emphasized playing more than learning. 
It was not difficult to understand that in 1980, only seventy-six primary schools 
representing about 1 0% of the total primary schools adopted Activity Approach. 

At present, Activity Approach is used in Key Stage One (primary one to three) of 
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slightly more than half of the primary schools in Hong Kong. It is seldom adopted in 
Key Stage Two (primary four to six) of primary education because like the parents, the 
school heads also query the effectiveness of Activity Approach. Moreover, the 
traditional approach is believed to be effective in preparing pupils for the Secondary 
Schools Places Allocation, a public examination at the end of the primary level for the 
purpose of getting into the secondary schools. 

Purpose of the study 

The implementation in the recent years of the Target-Oriented Curriculum 
conceptualizes that learning is an active, holistic, purposeful process of constructing, 
using and reconstructing knowledge (Clark et al, 1994). Moreover, knowledge is 
constructed rather than transmitted, and people learn through interaction between 
thoughts and experiences rather than passively reacting to them (Cheung, 1996). Thus, 
it is evidenced that the traditional way of learning by listening to teacher talk, 
answering questions and doing seatwork is ineffective to cope with curriculum 
innovation. 

The recent proposed changes in educational aims further reflect the inadequacy of the 
teacher-centred method of teaching in cultivating the kind of people who are creative, 
knowledgeable and multi-talented for Hong Kong to increase its competitiveness and 
continue its prosperity. The Education Commission (1999) proposes the overall aims 
of education are “to enable everyone to develop to their full potential in all areas 
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covering intellect, and social skills, so that each individual is ; filled with self- 

confidence and team spirit” (p. 15). Moreover, emphasis is laid on catering for 
individual differences, fostering students’ positive attitude and increasing their 
motivation as well as creativity. 

The scenario described above seems to support a pupil-centred approach in classroom 
teaching in order to bring about the kind of pupil performance that falls in alignment 
with the educational aims for the 21 st century. Cooperative learning, with its emphasis 
on academic and interpersonal skills as well as its use of individual differences to 
enhance learning (Johnson & Johnson, 1999), has great potential to be adopted as an 
appropriate instructional practice to help achieve the new educational aims. 
Cooperative learning is “the instructional use of small groups so that students work 
together to maximize their own and each other’s learning” (Johnson & Johnson, 1999, 
p. 5). Cooperative group work is excellent to be used in mixed-ability classes to help 
students learn to think, to solve problems, to integrate and apply knowledge and to 
increase their self-esteem (Slavin, 1995). However, the kinds of group work done at 
schools adopting Activity Approach have often been criticized as the kind of work that 
can be completed by individual pupil without the need of cooperation among members 
of the group. Furthermore, the situation of teachers using group work in schools 
which adopt traditional teaching approaches can only be speculated due to a lack in 
research data. 

The aim of this paper is three-fold. First, it begins with a review of the literature on 
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cooperative learning to form a setting for the later assessment of teachers’ attitudes. 
Second, it attempts to assess teachers’ attitudes towards the use of group work, such as 
their perceptions of the frequency of using it, their reasons for or against their choice of 
it and the organization of group task. Third, the attitudes of pupils towards working 
collaboratively will also be assessed. 

The preceding paragraphs set out the context of the use of group work in Hong Kong 
primary schools as well as the purpose of the present paper. Five main sections follow. 
The first section reviews the literature on cooperative learning. The second section 
describes the research method used for obtaining the relevant data. The third section 
describes and discusses the findings. The fourth section concludes the paper while the 
fifth section discusses the limitations of the study and comes up with some suggestions 
for later research. 

Literature review on cooperative learning 

Much research has been done on cooperative learning in the UK, USA and Israel in the 
past thirty years. According to Johnson and Johnson (1992), over 550 experimental 
studies and 100 correlational studies were done, the majority of which suggested that it 
had a positive effect on enhancing students’ performance in both the cognitive and 
social aspects of development (Johnson & Johnson, 1999). These research studies 
were guided by theories which are further validated by the research studies. 

The theoretical bases of cooperative learning encompass theories from psychology and 
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sociology. The major psychological theories include the Cognitive-Developmental 
Theory, the Cognitive Elaboration Theory, the Behavioural Learning Theory and the 
Humanistic Learning Theory. The sociological theories include the Social 
Interdependence Theory and the Contact Hypothesis. 

Cooperative learning methods 

Based on the theories listed above, various cooperative learning methods have been 
developed over the years and put into actual practice in the classroom. According to 
Slavin (1995), some of the mostly extensively researched and widely used methods 
include Student Teams- Achievement Divisions (STAD), Teams-Games-Toumaments 
(TGT), Jigsaw II, Team Accelerated Instruction (TAI), and Cooperative Integrated 
Reading and Composition (CIRC) (Slavin, 1995). Each of these methods has its own 
characteristics and applicability to different curriculum areas and students in different 
stages of learning, but they all possess some of the basic elements of cooperative 
learning: team rewards, individual accountability and equal opportunity for success. 
Some of the other popular cooperative learning methods include Group Investigation, 
Learning Together, Complex Instruction, and Structured Dyadic Methods. 

Basic elements of cooperative learning 

Little consensus is reached among various researchers concerning the basic elements of 
cooperative learning. According to Slavin (1990) and Kagan (1990), the basic 
elements of cooperative learning include positive interdependence and individual 
accountability. Johnson and Johnson (1986) put in three more elements including 
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face-to-face promotive interaction, interpersonal and small group skills, and group 
processing (Antil et al, 1998; Johnson & Johnson, 1999). Cohen (1994) views 
cooperative learning from another perspective and focuses on tasks and interaction 
instead. The following paragraphs discuss some of the basic elements of cooperative 
learning. 

Positive interdependence 

Positive interdependence is considered as the first and most important element of 
cooperative learning, enabling students to recognise that their goals can only be 
attained when the goals of all members in the group are also attained (Johnson et al, 
1990; Johnson et al, 1993). Consequently, in order to reach their common goal, 
members have to try their best to study the learning materials on the one hand, and to 
use every possible way to make sure that every member in the group has also 
understood the materials on the other. Furthermore, the members must realise that 
each one of them can make a unique contribution to the success of the group, then they 
will increase their efforts (Harkins & Petty, 1982). Otherwise, they will decrease their 
efforts if their potential contribution is regarded as dispensable (Kerr, 1983). 

According to Johnson and Johnson (1999), positive goal interdependence is the most 
important among the various types of positive interdependence as it reminds the 
members that their goal attainment depends on the goal attainment of the other 
members of the group. If any member of the group fails, little hope will be left for the 
success of the rest of the group members to be realised. Various types of positive 
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interdependence have been forward to supplement goal interdependence. They 
include positive celebration/reward interdependence, positive resource interdependence, 
positive role interdependence, positive identity interdependence, environmental 
interdependence, positive fantasy interdependence, positive task interdependence and 
positive outside enemy interdependence. 

Positive celebration/reward interdependence exists when each member of the group 
receives the same reward when they successfully complete a joint task. They then 
celebrate their success. Positive resource interdependence exists when each member 
of the group has a part of the information or materials, the whole of which is required 
for group to achieve its goal. Positive role interdependence exists when the group 
members are assigned complementary and interconnected roles such as leader, recorder 
and checker. It gives each member a responsibility to work towards the joint task. 
Positive identity interdependence exists when a mutual identity of the group is 
established by creating a name, motto or mascot for the group. Environmental 
interdependence exists when the group members gather together in close proximity to 
work. Positive task interdependence exists when the joint task is so designed that the 
completion of a member’s work depends on the completion of each other’s work. 
Positive outside enemy interdependence exists when the groups are in competition with 
one another. 

According to Johnson and Johnson, the above items of positive interdependence serve 
as supplement to the positive goal interdependence. For example, when either 
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positive resource interdependence or positive reward interdependence is used together 
with positive goal interdependence, both individual achievement and group 
productivity are increased (Johnson & Johnson, 1990). A further example can be seen 
from the studies (Lew et al, 1986; Mesch et al, 1986) which indicate that student 
achievement can be enhanced by using positive goal interdependence, but the effect is 
better when coupled with reward interdependence. 

Having said that, there are some researchers who do not agree with the Johnsons’ view 
of placing special importance on positive goal interdependence to explain the 
relationship between cooperation and achievement. Cohen (1994) and Slavin (1983) 
argue that positive goal interdependence alone may not be able to motivate the group 
members to interact with one another, resulting in low achievement. This may happen 
especially when the task is divided into parts and each member is responsible to work 
on one part on their own before compiling into a single group end product. Similarly 
there will also be little interaction among the group members if the single group task 
can be completed by one member. 

Hays and Slavin propose that positive reward interdependence gives a better 
explanation of the relationship between cooperation and achievement than positive goal 
interdependence (Sharan, 1990). After reviewing a number of studies which compared 
the effectiveness of cooperative learning with traditional learning with respect to 
student achievement, Slavin (1983) concluded that reward interdependence and 
individual accountability were the two important elements when used together in 




8 



9 



cooperative learning would enhance achievement. Slavin (1995) later suggests the 
following three important elements in his cooperative learning methods such as STAD, 
TGT and Jigsaw II: team rewards, individual accountability and equal opportunity for 
success. 

Despite the importance of positive reward interdependence towards achievement, 
Cohen (1994) proposes that if the group members are engaging in a true group task, 
reward interdependence may not be necessary for achievement so long as individual 
accountability is maintained. A true group task is usually an open-ended discovery or 
conceptual task characterised by an ill-structured problem. Cohen (1994) defines a 
group task as “a task that requires resources (information, knowledge, heuristic 
problem-solving strategies, materials, and skills) that no single individual possesses so 
that no single individual is likely to solve the problem or accomplish the task objectives 
without at least some input from others” (p. 8). For example, some cooperative 
learning methods like Group Investigation and Complex Instruction do not require 
reward interdependence but still are able to enhance achievement. Cohen argues that 
in Complex Instruction, the task is intrinsically interesting to motivate the group 
members to interact with one another to bring about achievement. Nevertheless, 
reward interdependence is needed for those group tasks which are actually 
collaborative seatwork in disguise as they can be completed by one member alone to 
motivate the members for interaction. 
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Individual accountability 

Individual accountability means that the success of a group depends on the individual 
learning of all the group members (Johnson & Johnson, 1989; Slavin, 1995). Apart 
from responsibility for one’s own learning, each member has also to be responsible for 
facilitating the learning of the rest of the group members. Individual accountability 
exists when the performance of each individual member is assessed, the results are 
given back to the individual and the group to compare against a standard of 
performance, and the member is held responsible by groupmates for contributing his or 
her fair share to the group’s success (Johnson & Johnson, 1999). As such, individual 
accountability motivates the group members to help one another to give maximum 
effort to learn (Slavin, 1995) so that when the member is assessed individually, each 
one can contribute their efforts to the success of the group. 

When individual accountability is not felt by the group members, “social loafing” may 
result meaning that for additive task in which individual members’ effort are summed 
up towards the group product, it may happens that only a few members of the group are 
actually working on the task, the rest of the group contribute a little effort without being 
noticed (Latane, Williams, & Harkins, 1979). Experiments conducted by Williams, 
Harkins and Latane (1981) confirm that when it is difficult to identity the contribution 
of the members, they will make less effort towards the group goal. Therefore, it is 
important to assess the group according to the individual learning of each member so as 
to structure individual accountability for maximum effect of cooperative learning 
(Manning & Lucking, 1991; Slavin, 1989). 
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Individual accountability can be structured by using explaining, oral examination and 
testing. Teachers can require all group members to explain or teach what they have 
learned to help each other learn in order to master the learning materials (Johnson et al, 
1993). Teachers can also conduct random oral examinations by selecting a member of 
each group as representative to answer questions for his or her group (Kagan, 1 990). 
Finally individual accountability can be structured by testing the group members so that 
individual group member has a clear picture of the level of each other for them to 
improve on through peer assistance (Johnson & Johnson, 1999; Slavin, 1995). To 
facilitate the structuring of individual accountability, it is advisable to start with small 
groups (Messick & Brewer, 1983). 

Face-to-face promotive interaction 

Engaging in cooperative learning, group members have to meet each other to work on 
the task. During face-to-face interaction, members encourage and facilitate each other 
in order to accomplish the task to reach the group goal. As Johnson and Johnson 
(1999) state, face-to-face promotive interaction among individuals fostered by the 
positive interdependence greatly influences efforts to achieve caring and committed 
relationships, and psychological adjustment and competence. Slavin also agrees the 
effect of face-to-face promotive interaction by citing the findings of Meloth & Deering 
(1992) in that students’ interaction in cooperative groups can be effective though there 
is no group rewards. Nevertheless, group interaction together with group rewards 
yields much better outcomes than either one alone (Fantuzzo et al, 1992). 
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The relationship between interaction and achievement, however, is not so simple as 
what has been described in the preceding paragraph. Research studies based on the 
simple frequency of interaction as a predictor for achievement had inconsistent results. 
Webb (1983) found that there was little relationship between interaction and 
achievement, but Cohen, et al (1989) found that simple measures of frequency of task- 
related interaction were related to achievement. Cohen analysed these studies and 
proposed that positive correlation between interaction and achievement or productivity 
depended on the nature of the group task. In Webb’s studies, the tasks which were 
solved in well-structured methods and had a right answer could actually have been 
completed by individuals. In the study of Cohen, et al, the tasks which were open- 
ended having an ill-structured problem could only be accomplished by the cooperation 
of the group members, each of whom had some indispensable resources such as 
knowledge, skills and materials. Cohen (1994) concludes that “given an ill-structured 
problem with no right answer and a learning task that will require all students to 
exchange resources, achievement gains will depend on the frequency of task-related 
interaction” (p. 8). However, the more developmentally advanced student in a 
heterogeneous group working on ill-structured problems may be cognitively affected in 
the course of interaction with the less developmentally advanced student (Tudge, 
1990). 

For Webb’s studies of group tasks which can be solved by individuals, the giving of 
detailed explanations rather than the measures of interaction determines students’ 
achievement (Webb, 1983). Also, it is the student who gives explanations achieves 
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more than the student who receives the explanations. However, if the student who 
asks for explanations is only given the correct answer without being explained, the 
more he asks for help, the less he achieves (Webb, 1991). 

In an experiment to compare the learning outcomes using STAD and Group 
Investigation, it was found that the STAD classes performed better on low-level 
questions while the Group Investigation classes performed better on high-level 
questions (Sharan et al, 1984). Cohen (1994) concludes that the experiment illustrates 
the association of learning outcomes with the kind of interaction brought about by the 
kind of tasks and task instructions. As a conclusion, the amount and type of 
interaction that affect achievement differ according to the nature of the task. 
Consideration has to be given to the kind of learning outcome expected when 
structuring interaction. 

Interpersonal and small group skills 

Social skills are the key to the productivity of a group (Johnson & F. Johnson, 1997) as 
these skills reduce interpersonal conflict and facilitate interaction (Cohen, 1994). 
Studies show that if the students are awarded the using of social skills, they will 
become more sophisticated in using the skills and will use the skills more often 
resulting in a higher achievement (Lew et al, 1986). 

There are a variety of interpersonal and small group skills including for example, 
moving about quietly, monitoring time, interrupting appropriately, encouraging one 
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another and resolving a conflict. These skills do not come automatically with 
cooperative learning (Barnes & Todd, 1977), but have to be explicitly taught as 
systematically as any subject like mathematics or social studies in order for the groups 
to be productive (Johnson & Johnson, 1989; Johnson et al, 1993). To teach these 
skills, the following steps have to be followed: 

First, students must see the need to use the skill. Second, students must 
understand what the skill is and when it should be used. Third, to master a 
social skill, students must practise it again and again. Fourth, students must 
process how frequently and how well they are using the skills. Fifth, students 
must persevere in practising the skill (Johnson & Johnson, 1990, p. 30). 

Group processing 

Group processing according to Johnson and Johnson (1990) is defined as “reflecting on 
a group session to describe what member actions were helpful and unhelpful, and make 
decisions about what actions to continue or change” (p. 32). Through group 
processing, interpersonal conflict is reduced, the probability of desired behaviours to 
complete the task, and members caring for one another increase, resulting in a highly 
motivated group (Cohen, 1994). 

In a study conducted by Yager, comparison was made as to achievement between 
cooperative learning group with group processing, cooperative learning group without 
group processing, and individual learning group. It was found that the cooperative 
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learning group with group processing performed the best on daily achievement as well 
as retention measures (Yager et al, 1985). A follow up study was conducted to 
compare the achievement between cooperative learning group with teacher processing, 
cooperative learning group with teacher and student processing, cooperative learning 
group without processing, and individual learning group. The results showed that the 
cooperative learning group with teacher and student processing had the highest scores 
(Johnson et al, 1990). 

Two conditions have to be met before group processing can have effect on achievement. 
The first condition is that the behaviours to be processed must be specific (Huber & 
Eppler, 1990). The second condition is that the behaviours to be processed must be 
directly relevant to the desired behaviours in the task (Miller & Harrington, 1990). If 
these conditions are satisfied, processing the group while the members are working on 
the task can also raise the productivity of the group. 

Associating the basic elements of cooperative learning with its theoretical bases, it can 
be postulated that the framework of cooperative learning as viewed by Slavin and 
Johnson & Johnson, with emphasis on positive interdependence and individual 
accountability, is based largely on the psychological theories as described earlier. 
However, Cohen (1994) focuses her attention on the nature of the tasks and interaction, 
and is perceived to have a strong link with the sociological theories (Antil et al, 1998). 
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Method of the study 

The study basically employed the survey method of the non-experimental design for 
data collection since a survey deals with how people feel or perceive (Wiersma, 1986). 
Different instruments were used to collect data from the subjects. For the teachers, 
they were asked to fill out a questionnaire followed by a semi-structured interview. 
For the pupils, the data was obtained by using the method of projection. 

Subjects 

Primary school teachers and the pupils they taught were chosen as a sample for this 
study. As the study was intended to be the pilot study of the final thesis, the samples 
were small, containing five teachers and sixty-five pupils from a bisessional school. 
The teachers were conveniently selected from a school of average standard near the 
place where the author lives. The standard of the school was determined by their 
pupils’ performance in the Secondary Schools Entrance Allocation which is a system of 
selecting and allocating the pupils according to their academic abilities to the secondary 
schools after the six-year primary schooling. 

The teachers all taught the upper primary classes in the school. The choice for the 
upper primary class teachers was made having considered the age of their pupils. If 
the pupils were too young, they might have difficulty in responding to the instrument 
which required them to write. The teacher population of the study was 1 2 from which 
a sample of 5 was obtained by random sampling. The pupil population was 183 which 
was the total number of pupils in the five classes of the teacher sample. Out of the 
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1 83 pupils, a sample of 65 was obtained by random sampling so that the sample thus 
obtained had a high probability of being representative of the population from which it 
is drawn. 

Questionnaire 

The questionnaire for the teachers was adapted from the questionnaire entitled “Survey 
of Primary Classroom Organisation” which was designed by the National Institute of 
Education. The questionnaire was divided into two parts. Part I which was the first 
section asked for the personal information of the respondents. Part II, comprising 
eight sections, focussed on the organisational strategies used by the respondents in their 
classrooms. The second section dealt with the frequency of using classroom 
organisational strategies in four core subjects. The third section dealt with the basis of 
grouping for small group work. The fourth section looked at the size of groups. The 
fifth section sought information on how the group tasks were organised. The sixth 
section dealt with the social skills of working in groups. The seventh section asked 
the respondents to write down the factors which might influence their decision to use 
the various classroom organisational strategies. The eight section asked the 
respondents to describe a successful and another unsuccessful use of group work in 
their teaching. The final section asked the respondents to write down some of the 
problems which they might encountered in using group work in their classes. 

Interview 

Four respondents agreed to take the interview immediately after finishing the 
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questionnaire while one respondent promised to do it in the following day. The 
interviews took place in the school where they worked. This made the interviewees 
feel secured and have a tendency to express their views in greater depth. The 
interviews ranged from three-quarter of an hour to one hour and a half. Recording of 
the interviews was not done as the interviewees were uncomfortable to it. However, 
they compensated the author with more in-depth discussion though the flow of the 
interview was sometimes blocked by taking some long notes on paper. 

Projection 

Problems might exist if respondents’ attitudes were measured by using questionnaires 
as effectiveness of attitude scales depend on the cooperation and frankness of the 
respondents (Oppenheim, 1992). For example, respondents cannot give the genuine 
answer just because they themselves are not aware of their own attitudes. Sometimes, 
in order to live up to the expectations of their society, the respondents may have 
reservations in disclosing their inner self. There are times that respondents will give 
the answer which they feel the researcher wants. 

In order to minimise 

the phenomenon known as ‘agreed response bias’ where the respondents 
provide answers which they feel the designer of the questionnaire wants rather 
than those which they feel best expresses their own beliefs and values,.... 
much attitude research has concerned ways of encouraging respondents to 
reveal their real beliefs and values by creating situations where they were not 
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aware of the purpose of the exercise. In the case of primary-aged pupils, 
where the urge to please the teacher can be very strong, there were therefore 
dangers in using any kind of attitude inventory which asked questions about 
which classroom practice the children preferred (Galton & Williamson, 1994, p. 
60-61). 

Because of the shortcoming of the questionnaire in measuring children’s attitudes, the 
pupils’ attitudes towards learning collaboratively in the present study were measured by 
employing the method of projection. Projective techniques work on the belief that 
when a child is given a stimulus which is ambiguous or when the child is asked to 
discuss a picture in imaginary terms, he will response to it by drawing on his own inner 
feelings and needs (Oppenheim, 1 992). 

In the present study, the pupils were asked to respond similar to the Thematic 
Apperception Test, but in writing, to a pair of cartoon pictures. The cartoon pictures 
which were copied from the book “Group Work in the Primary School” written by 
Galton and Williamson (1994) were about four children discussing into a tape recorder 
at the centre of a table, one with the teacher present, and another with the teacher absent. 
Each pupil was asked to write down what he thought the children were saying or 
thinking on the lines provided below the pictures. The pupil could pick any two of the 
children to write on. Finally, the pupils were asked to make a choice on three faces: a 
smiling face, an expressionless face and a sad face depicting a preference for group 
work, no idea, and non-preference for group work. The adding of the element of 




20 



19 



expressing preference for one situation rather than another was significant as it helped 
to ease the problem of determining motivating attitudes, the predictors of action, as 
raised by Arnold (Gabon & Willaimson, 1994). 

Results and Discussion 

The survey was mainly designed to find out some facts about teachers’ and pupils’ 
perceptions towards cooperative group work such as the frequency of using group work 
and the size of the groups. It was less designed to explain the phenomenon, for 
example, why the teachers and pupils had such kinds of perceptions. The results and 
discussion that followed would largely be on describing the present phenomenon 
except the interview which tried to look at the phenomenon in greater depth. 

Teachers’ and pupils’ demographic data 

The five teachers had substantial teaching experience: one finished teacher training in 
the 1970’s while the other four finished teacher training in the 1 980 ’s. The school 
operated two sessions and the teachers only taught in the AM session. One teacher 
was specialized in teaching Mathematics and English while the remaining four teachers 
all taught Chinese, English and General Studies. The sixty-five pupils aged from 9 to 
12. They had all studied in the school since primary one and they came from similar 
families of low social economic status. 
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Organisational strategies 

The following table summarised the frequency of group work used by the teachers in 
different subjects. 

Table 1 Group work used in different subjects (adapted from the Survey of Primary 
Classroom Organization designed by the National Institute of Education, Singapore 
and the School of Education, Leicester University, UK.) 



never 


seldom 


sometimes 


often 


almost always 


(0% of 


(25% of 


(50% of 


(75% of 


(90% of the 


the time) 


the time) 


the time) 


the time) 


time) 


Chinese 


1 


3 






English 

Mathematics 1 




4 






General Studies 




2 


2 





Table 1 illustrates that all the teachers used group work at various frequencies for 
Chinese, English and General Studies except one teacher in Mathematics. The 
mathematics teacher indicated at the interview that whole class work was almost 
always adopted in teaching mathematics as they wanted to fully utilise the teaching 
time to enable the pupils to practise as many exercises as possible. Comparatively, 
more teachers engaged half of their teaching time in using group work. 

The teachers had attempted to use group work as a classroom practice in the subjects 
they taught. Two explanations could be offered. One might be that the teachers had 
been employing group work during the implementation of the Target Oriented 
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Curriculum which stressed the importance of working on tasks to accomplish the 
targets. Many of the tasks were contextually designed and required the pupils to work 
together to carry them out. Another explanation might be that the schools had adopted 
activity approach for some time, and the teachers had been customary in using group 
work in the lower primary level. Relatively speaking, group work was used more 
often in General Studies than in other subjects. Four teachers indicated that it was 
because of the topics or activities in General Studies that lent themselves to be more 
effectively carried out in the form of group work than other forms of organisational 
strategies such as whole class work, pair work and individual work. 



Basis of grouping 

Basis of grouping used by the teachers was summarised in the following table. 



Table 2 Basis of group work used in different subjects (adapted from the Survey of 
Primary Classroom Organization designed by the National Institute of Education, 
Singapore and the School of Education, Leicester University, UK.) 



Chinese 


English 


Maths 


General Studies 


Sitting near 


3 


4 




4 


Random 




1 






Similar ability 




1 






Different abilities 


1 


3 






Working well 




1 






Free choice 




2 




1 


Same gender 




1 






Opposite gender 


1 


1 






Same ethnic group 




2 






Different ethnic group 


1 


1 
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Despite the requirement of grouping pupils of different abilities in cooperative groups, 
this basis of grouping only ranked second in the present study. The teachers mostly 
preferred grouping their pupils who were sitting close to each other. All the teachers 
unanimously responded in the interview, 

This kind of convenient grouping does not only save time, but also produces 
the least noise. You have to be practical and feasible. Any other ways of 
grouping result in various degrees of chaos with which the precious teaching 
time cannot afford to spare. 

Research generally suggests mixed ability grouping will be more likely to bring about 
greater achievement outcome than other grouping strategies such as random assignment 
or convenient grouping (Lou et al, 1 996). 

It was remarkable that although the basis of grouping used in English were also used in 
General Studies, additional methods of grouping were used in English, such as different 
ability, working well, same gender and ethnic group. The four teachers explained, 

Group work for General Studies are interesting enough to motivate the pupils 
to participate irrespective of the methods of grouping. Then why not choose 
a convenient way that saves you time and trouble. However, with English, a 
subject that pupils find difficulty with, motivation to learn is critical for the 
success of the group work. Besides, some English activities require special 
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grouping such as same/different ethnic groups or same/different gender to 
accomplish. 

The teachers seemed to suggest that the way pupils were grouped would not be taken 
into consideration so long as the pupils were already motivated by the learning 
activities. They had overlooked one of the criteria for successful cooperative or 
collaborative group work — mixed abilities (Johnson & Johnson, 1994; Slavin, 1995; 
Vermette, 1998). 

Group size 

There was no significant difference in the group sizes that were most often used in 
Chinese, English and General Studies. (Mathematics was excluded as no group work 
was reported for the subject in this study.) Two teachers indicated a group size of 4-5 
while three teachers added a group size of 6-7. This was consistent with the data on 
grouping pupils sitting near to each other. Pupils of the five schools all sat in rows. 
A typical primary classroom accommodated 35 to 37 pupils. When they did group work, 
the easiest way to form a group was for the two pupils in front to turn their back to face 
the two pupils sitting behind, resulting in a group of 4. It was further evidenced by the 
teachers during the interview. One teacher replied, 

I have 35 pupils in my class. When we do group work, we form 5 groups of 4, 
and 3 groups of 5 by grouping the pupils sitting in the neighbourhood. 

Another teacher said, 
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There are 37 pupils in my class. It is both natural and convenient to form 8 
groups of 4, and 1 group of 5. 

The other three teachers preferred bigger groups of 6-7 considering the factors of 
limited space and resources of the classroom. 

The group size of 4-5 is similar to that used in cooperative learning methods such as 
STAD, TGT and TAI. According to Slavin (1995), a four-person STAD team should 
represent a cross-section of the class, comprising two boys and two girls who can also 
be classified into one high performer, one low performer and two average performers. 
Moreover, most of the successful studies researching on cooperative learning used 
teams of four (Vermette, 1998). 

The group size of 6-7 may also be found in cooperative learning, but is not common. 
For example, Blaney, et al (1977) found that groups of six were successful in Jigsaw. 
However, with group size more than five, the possibility of becoming a “free rider” is 
greater, and it is more difficult to come to a consensus during discussion. 

Organisation of group task 

Various forms of organisation of group tasks were listed in the questionnaire for the 
teachers to indicate which forms they had used with their pupils. The results were 
summarised in the following table. 
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Table 3 Organisation oJLgroup tasks used (adapted from the Survey of Primary 
Classroom Organization designed by the National Institute of Education, Singapore 
and the School of Education, Leicester University, UK.) 



Organisation of group tasks 


Teachers 
Use Not use 


Pupils individually contribute sentences to a joint piece of 
written work. 


5 


0 


Pupils jointly discuss their ideas in response to teacher’s 
questions and present them orally as a group. 


4 


1 


Pupils jointly discuss questions and one pupil is called upon 
at random to answer on the group’s behalf. 


4 


1 


Pupils jointly discuss their ideas and write a group report. 


0 


5 


Pupils jointly discuss their ideas and each group member 
writes a section of the group report. 


3 


2 


Pupils discuss a worksheet together and submit completed 
worksheets individually. 


4 


1 


Pupils divide different aspects of the group task and work 
on their own sections individually 


3 


2 


Pupils study a section of a chapter in pairs, then explain 
what they have learnt to their group members. 


1 


4 



The result indicated that the teachers had attempted all the eight group tasks except the 
task in which pupils were required to jointly discuss their ideas and write a group report. 
The interview with the teachers seemed to suggest that it was more difficult for the 
pupils to jointly write a group report after joint discussion as they did not feel confident 
to write. Moreover, the teachers had reservations in letting pupils study a topic and 
then teach their group members. 



The seven group tasks which the teachers attempted had some of the basic elements of 
cooperative learning such as positive interdependence in terms of goal, resource, role, 
task and environmental, individual accountability and face-to-face promotive 
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interaction. For example, the task “Pupils jointly discuss their ideas in response to 
teacher’s questions and present them orally as a group” requires positive goal 
interdependence and positive role interdependence. The task “Pupils study a section 
of a chapter in pairs, then explain what they have learnt to their group members” 
requires individual accountability. However, the tasks “Pupils individually 
contributes sentences to a joint piece of written work” and “Pupils divide different 
aspects of the group task and work on their own sections individually” are in fact 
collaborative seatwork that can be completed by one member of the group. These two 
kinds of tasks had been chosen by a majority of the teachers. 

Social skills 

Social skills are necessary for the success of group work (Johnson et al, 1993). The 
teachers all agreed that there was a need for their pupils to learn the following social 
skills: moving about quietly, staying with the group, speaking in quiet voices, using 
names and eye contact, monitoring time, staying on the task, taking turns, listening 
actively, contributing ideas, asking questions, interrupting appropriately, encouraging 
one another, criticising ideas, not people, sharing feelings and resolving a conflict. 

The teachers also admitted that they sometimes taught their pupils these social skills, 
but they thought it might be better for the pupils to learn the social skills incidentally 
instead of doing it during the lesson itself or outside lesson time. They believed that 
these social skills had to be developed over time and in a natural setting, not necessarily 
restricted to inside school. One of the teachers suggested that these skills should be 
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treated like the hidden curriculum of a school. However, in order for group work to 
be effective, Johnson et al (1994) suggest it is better for the teachers to teach the social 
skills required for the task at the beginning of the lesson. For effective cooperative 
learning, social skills had to be taught (Smith, 1987). Only a few studies for example, 
suggested that explict teaching of collaborative skills had no effect on student 
achievement (Slavin, 1995). 

It was found that most of the teachers in this study never encouraged their pupils to 
evaluate how well they worked together as a group, or their individual contribution to 
the group. These skills are referred to group processing skills in cooperative learning 
and should be practised in order to increase the effectiveness of the group (Schultz, 
1989). 

Furthermore, although the teachers admitted the importance of team spirit, non had 
created a name, motto or mascot for the groups because they doubted the effect of it on 
the promotion of team spirit. They had underestimated the effect of a motto or mascot 
on structuring positive identity interdependence (Johnson & Johnson, 1999). 

Teachers attitudes towards group work 

In general, the teachers’ attitudes towards group work were positive. One of the 
teachers recalled, 

When I was a novice teacher some twenty years ago, my lesson was teacher- 
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centred. Very soon the pupils felt bored and did not pay attention. In the 
end, it was the teacher who seemed to enjoy the lesson. Later I introduced 
group work into the classroom, the whole atmosphere changed. It activated 
the learning process of the pupils as the pupils began to realise that it was 
they themselves who were responsible for their learning, and sometimes also 
responsible for their groupmates learning. Thus, learning became pupil- 
centred. Pupils were no longer seen taking dozes during the lesson. 

This was also echoed by the other four teachers. The five teachers could perceive the 
significance of individual accountability in group work. 

One teacher found that the weaker pupils had improved their academic performance. 

Allocate the weaker ones to the groups of brighter pupils and see how they 
show remarkable improvement over a short time. I come to believe that the 
weaker pupils can understand their peers’ explanation of the text better than 
mine. It seems that they have a different language. 

Three teachers also had the same opinion as this teacher. The results support the study 
of Antil et al (1998) in which it was found that academic learning and active 
involvement were two of the rationales for teachers’ subscribing to cooperative 
learning. In other words, the majority of teachers could perceive the importance of the 
mixed ability grouping in cooperative group work. Van Oudenhoven, et al (1987) 
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found that low achievers gained most in cooperative learning. In some studies, the 
high achievers were found to be the ones who benefit most from cooperative learning 
(Webb, 1989). In fact both the high and low achievers gain more than their 
counterparts in control groups (Sharan, et al, 1984). However the results of the basis 
of grouping in the present study suggested that the teachers preferred convenient 
grouping unless they wanted to increase the motivation of their pupils. This reflected 
the constraint they had in grouping. 

One teacher had a different view. 

There are two conditions that have to be satisfied before the weaker pupil can 
benefit from group work. First, he must gain the acceptance of the bright 
pupils in the group; otherwise, he becomes an alien and psychologically suffers. 
Second, he is not too influential to change the whole group into a play group. 

Research on cooperative learning provides evidence that the above two conditions have 
been well taken care of. Successful cooperative groups are generally made up of 
members of mixed abilities and different ethnic groups. Members of different ethnic 
groups come to know and respect one another better after staying in cooperative groups 
(Sharan, 1980). Face-to-face promotive interaction facilitates the caring of each other. 
Moreover, the elements of positive role interdependence, especially the goal and role 
interdependence, and individual accountability that are built in cooperative learning 
help to prevent the condition of over-dominance of a certain member in the group from 
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happening. 



Two of the teachers shared their experience of cooperative groups. 

Yes, we teachers know the values of cooperative groups and how both the 
bright and the weak pupils benefit. However, the parents don’t understand. 
They keep asking you why their children learn from their better peers instead 
of me. Is it that the peers know even better than the teacher? Or is it that 
the teacher has found another excuse for his laziness? There are also cases 
in which the weaker pupils are made to run errands for the brighter peers as a 
return for their tuition. Sometimes it is the parents of the brighter pupils 
who come to ask you why their children have suddenly become teachers. 
They worry that their children may have been exploited. 

As mentioned earlier, in doing group work, both the brighter and weaker pupils benefit. 
The brighter pupils improve their metacognition as they teach the weaker pupils who, 
as a result, can come to understand the thing in question better. The rather negative 
feelings of the teachers and parents are not difficult to understand (Antil et al, 1998) as 
cooperative learning which stresses the responsibility of learning on the part of the 
pupils is new to them. Furthermore, the eastern culture has taken the concept of 
teacher-centred learning for granted for a long time and regarded the transmission of 
knowledge rests with the teacher. 
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One of the teachers said, 



Frankly speaking, I personally do not like to use group work though it may be 
good to the kids. There are so many constraints. For example, we have a 
modest classroom but with 37 pupils who can drive you mad. You cannot 
request them to sit still with mouth shut during group work. But who knows 
what the headmaster thinks when he walks by hearing the noises? Also, can 
learning really take place under such a noisy classroom? 

Chinese people often think that a class of good discipline is a quiet class in which 
effective learning takes place. The teacher’s worry of the effectiveness of learning in 
a noisy classroom reflected that the teacher knew little about or had little confidence on 
cooperative learning. Research suggests that children are motivated to learn in 
cooperative groups, and knowledge is constructed in the interaction of the peers. 

Three of the teachers pointed out another constraint. 

Group work is fine after examinations when there is nothing left in the 
textbook to teach. Otherwise, with such a tight teaching schedule but 
limited time, you can’t afford the luxury of group work. May be it can be 
done once in a while when a certain topic requires plenty of sharing among 
the pupils. 
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Learning in cooperative groups takes a longer time than learning by listening to the 
teacher. But since the former is more effective, the memory retention is better. 
Moreover, children learn faster when they are highly motivated in cooperative groups. 
When the children become active in their learning, it helps to ease the problem of 
having a tight teaching schedule. 

Having analysed the teachers’ data from the questionnaire and the interview in the light 
of cooperative learning, it seems to suggest that their reservations in using cooperative 
group work may be due to their lack of knowledge in this strategy. 

Pupils’ response to the cartoon pictures 
Abstract talk versus information-seeking talk 

Only five children had the conversation centering on asking for information what to do 
next. 

Pupil 1 : What are we going to do? 

Pupil 2 : 1 don’t know. 

Two children were concerned on confirming what they have done is correct. 

Pupil 1 : Is this answer correct? 

Pupil 2: Yes, you are right. 
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These two types of conversation were not significant in the present study in view of the 
small number of pupils who reflected this. 

Examples of abstract talk reported were as follows: 

Pupil 1 : Let’s start with the problem of traffic congestion. 

Pupil 2: A good idea. 

Pupil 1 : I think the reason why the roads are so congested is that people do not like to 

take public transport. 

Pupil 2: I think it is because the roads are often dug up for repair. 

Pupil 3: Our health is being threatened by the drivers who use diesel or leaded petrol. 

Pupil 4: Not as much affected as the traffic accidents caused by careless drivers. 

Pupil 3: I suggest the government should enforce all cars be driven by unleaded 

petrol. 

Pupil 4: I suggest to impose heavy penalty on the reckless drivers. 

The above two conversations illustrated a kind of cooperative activity “in which the 
pupils, although working on a common task, nevertheless prefer to produce individual 
responses rather than a joint outcome” (Galton & Williamson, 1994, p.86). 

The following reported conversation illustrated another kind of cooperative activity. 




35 



34 



Pupil 5: There should be more policemen to check the cars which are speeding. 

Pupil 6: I know they have special cameras to take the photos of the cars as evidence 

to bring an accusation against the drivers. 

Pupil 5: Sometimes the plain-clothes will drive a private car with the camera installed 

in to check the speed of the cars passing by without being noticed. 

Pupil 6: Then the government should recruit more policemen to carry out this work. 

The pupils in the above conversation “appear to be involved in collaborative activity 
engaging cooperatively on a task with a joint outcome in mind” (Galton & Williamson, 
1994, p.86). 

The reported conversations quoted so far were conducted in the teacher’s presence. 
Conversations conducted in the teacher’s absence were quite similar, with the majority 
on abstract talk suggesting the children were fond of this kind of discussion activity 
irrespective of the teacher’s presence. 

The table below summarised the number of the kind of talk the children were 
conducting with and without the teacher’s presence. 

Table 4 Distribution of types of conversations with and without the teacher’s 
presence 





Abstract talk 


Information-seeking talk 


Teacher’s presence 


50 


15 


Teacher’s absence 


47 


18 
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As the two sets of figures were similar, it might be suggesting that the element of 
teacher was not a factor affecting the kinds of conversation the children were 
conducting. Analysing the teacher’s talk seemed to provide an answer to this finding. 
More than half of the teacher’s talk was positive, for example, giving praise and 
offering help. 

Pupils’ preference for collaborative group work 

Children were asked to indicate their attitude towards collaborative group work by 
choosing among a smiling face, an expressionless face and a sad face. The result was 
shown in table 5. 



Table 5 Attitudes towards collaborative group work 



Number of children 


Fond of group work 


28 


No idea 


30 


Not fond of group work 


7 



The number of children who expressed a preference for collaborative group work was 
four times as many as the children who did not like group work, and was nearly the 
same as the number of children who could not make up their mind. This finding was 
interesting when compared with the finding in Table 4. For example, the number of 
abstract talk and information-seeking talk (with the teacher’s presence) were 50 and 15, 
respectively. Assuming that children who conducted abstract talk had a more positive 
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attitude towards group work, the two sets of figures should equate. In other words, 
there should also be around 50 children who were fond of group work instead of 28 
children. Similarly, there should also be around 15 children who did not like group 
work instead of 7 children. The difference in the number of children in these two 
categories might have been affected by the 30 children who had chosen ‘no idea’. 
Moreover, more children (22) though indifferent to the group work, had chosen to 
conduct abstract talk suggesting that they might be more inclined to favour group work. 
Relatively fewer children (8) though indifferent to the group work, had chosen to 
conduct information-seeking talk suggesting that they might be more inclined to dislike 
group work. 

The following argument might be put forward to explain the discrepancy in the two sets 
of figures obtained. When students were asked to express their views towards a 
certain issue, they tended to respond in the direction in which they speculated what 
their teachers wanted them to in order to please the teachers. In the present study 
when the pupils were asked to indicate their preference for or against group work, 
nearly half of the pupils (30) had chosen ‘no idea’ because they simply could not guess 
what the teacher wanted them to answer. Choosing ‘preference for’ might be as 
dangerous as choosing ‘preference against’. However, when these 30 pupils were 
asked to respond to a pair of cartoon pictures, they were more ready to show their true 
feelings as they were not aware that by doing so, their true feelings could be reasonably 
projected. Of the 50 pupils who exhibited elements of abstract talk, 28 had clearly 
indicated that they like group work, the remaining 22 pupils would have come from the 
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30 pupils who had chosen ‘no idea’. 



Conclusion 

The teachers’ attitudes towards group work in the present study was generally positive. 
Relatively more teachers spent half of their teaching time in group work. These kinds 
of group work showed elements of positive interdependence, individual accountability 
and face-to-face promotive interaction. Various cooperative group tasks and 
collaborative seatwork had been attempted. 

On the whole, the teachers had reservations in using group work because they were 
under many constraints at schools. First, the size and setting of the classroom made 
the conduct of group work difficult. Second, group work could only be hastily done 
to save time for completion of the tight curriculum. Third, the head of school might 
query the ability of the teacher in classroom management when the pupils inevitably 
produced noise during group work. Fourth, the parents might think that the teachers 
were lazy. 

Not all the teachers used heterogeneous grouping. Nor did they recognise the 
importance of explicitly teaching the pupils the necessary social skills for the group 
tasks. Furthermore, group processing was rarely conducted to enhance achievement. 
Therefore the teachers failed to satisfy Johnsons’ requirements for cooperative learning, 
i.e., the five basic elements of cooperative learning. 
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Furthermore, the results seemed to suggest that the majority of the pupils liked to learn 
by participating in collaborative group work though further data had to be collected 
from the pupils by interview to confirm their attitudes. However, the motivation of 
the teachers to use group work will not increase if the constraints still linger on. In 
Hong Kong, teachers’ influence on the curriculum and the size of the classroom is 
minimal. What seems to be an easier way (may be even more difficult) is for the 
teachers to open up their classrooms to the heads of schools and the parents in order to 
give them a better understanding of the value of cooperative group work, but the 
teachers have to first of all strengthen their knowledge and skills of using cooperative 
learning. When signs of improvement in the pupils’ performance and sight of the joy 
of pupils’ learning become apparent, the heads and the parents may come to understand 
the magic of cooperative group work. The alignment of the views of the teachers, 
pupils, school heads and parents will then help in the implementation of cooperative 
learning at school with a view to achieving the new educational aims for the 21 st 
century. 

Limitations and suggestions 

Most of the limitations arise from the fact that the present study is a pilot one with a 
small sample as well as that the questionnaire is not specifically designed in the Hong 
Kong context. 

In the analysis of the data, it was found that inconsistency sometimes existed among 
sections 1.6, 1.7, 2, 3 and 4. For example, a respondent had not circled Mathematics 
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in section 1.7 meaning that he did not teach the subject, but was found responding to 
section 2.1 about the frequency of using various classroom organisational strategies in 
teaching Mathematics, and also responding to section 3 and 4 about the basis of 
grouping in Mathematics and the size of grouping in Mathematics, respectively. This 
reflects a clearer instruction or a reminder has to be given to the respondents. 
Moreover, section 1.7 asking for the subjects taught regularly is not really needed and 
may be redundant as such kind of information can be obtained in sections 2, 3 and 4. 
The last line on page three asking for information on the year group had also caused 
trouble. Some respondents also gave answers inconsistent with those for section 1.6 
about the levels taught. All these problems suggest a more careful arrangement of the 
items either by combining or deleting some of the items from the questionnaire may be 
necessary. 

The questionnaire intends to help the respondents to answer the questions by asking 
them to respond to the questions based on the information of the past month as shown 
on the top of page 3. However, this may cause a problem, of getting biased responses. 
For example, a respondent might be trying out a new classroom organisational strategy 
in the past month after attending a seminar or workshop. The response he gave would 
relate very much to the new strategy in the past month, which was not the strategy he 
used to employ, resulting in biased information. It is suggested that respondents 
should be asked whether they are currently or have recently attended some workshops 
or courses on classroom organisational strategies so that inference might be drawn from 
the answers. Alternatively, the respondents can be asked to respond to the questions 
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based on the information in a longer duration of time such as in the last term or in the 
past six months so that a more reliable response can be obtained. 

Only one respondent attempted a response to the open-ended questions such as section 
7 which asked them the factors that might influence their decision to use the various 
classroom organisational strategies, and section 9 which asked them the problems 
encountered in using group work in their classes. But the answers were both brief and 
vague. The rest of the respondents did not write down anything there. Too many 
nonresponses may affect the reliability of the questionnaire. It is suggested that the 
open-ended questions can be revised in the format of the closed-ended questions for 
better response rate. For example in section 7, all the possible intervening factors can 
be listed out for respondents to choose or rank in terms of their relative importance. 
In this way, respondents may be more willing to respond. In case they find other 
factors which do not appear on the list, they can supply them by writing against the 
option “others”. 

This pilot study used a convenience sample of respondents from one school of average 
standard. The results cannot be generalised beyond this particular school. It is 
suggested that a further pilot drawing sample from different schools of various 
standards is necessary. The results may provide richer information and may also be 
compared with those in this study to check the reliability of the questionnaire. 

No recording was done during the interview at the teachers’ request. Although this 
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made the interviewees feel comfortable and perhaps could elicit more in-depth 
information, it lost the advantage of replaying the tapes for complete analysis of the 
exact wording and tone of the conversation. It is therefore suggested that a covering 
letter should be designed and sent together with each questionnaire depicting the 
purpose of the survey, the use of the data, and the anonymity of the respondents in 
order to gain their confidence in responding to the taped interview. 

Attempt had been made to put forward an argument to explain the discrepancy in the 
two sets of figures on the pupils’ preference for collaborative group work. However 
the argument has to be tested on its validity before it can be accepted. It is suggested 
that interviews should be conducted for the triangulation of the data obtained by the 
projective techniques. 

Finally, it must be stressed that the present study is only of small-scale and the result 
can only reflect the condition of the sample. For example, it was found that a majority 
of the pupils showed element of abstract talk in their conversations, and that the 
presence of the teacher had little effect on the kinds of talk. Whether the findings are 
due to a biasd sample have to be followed up in interviews with the pupils and tested 
out in other further research, larger in scale. ' 
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